Non-local self-similarity in natural images has been well studied as an effective prior in image restoration. However, for single image super-resolution (SISR), most existing deep non-local methods (e.g., non-local neural networks) only exploit similar patches within the same scale of the low-resolution (LR) input image. Consequently, the restoration is limited to using the same-scale information while neglecting potential high-resolution (HR) cues from other scales. In this paper, we explore the cross-scale patch recurrence property of a natural image, i.e., similar patches tend to recur many times across different scales. This is achieved using a novel cross-scale internal graph neural network (IGNN). Specifically, we dynamically construct a cross-scale graph by searching k-nearest neighboring patches in the downsampled LR image for each query patch in the LR image.

Weaknesses: As in many SISR papers these days, the metric increase in PSNR and SSIM is vanishingly small. The qualitative example shown in Fig 3 demonstrates a more significant boost in metrics, however, this is obviously not reflected in the datasets as a whole given the average values reported in Table 1. I think it would be valid to state that this method mostly only helps in regions of self-similarity; it would be interesting to see if there is a metric that could capture this though rather than relying on anecdotal crops. The difference from previous graph convolutions applied for image restoration (like in [34]) seems fairly minor. Even the module as a whole is effectively a pretty small change to EDSR, involving very few additional layers.

All four reviewers felt that the paper was above acceptance threshold. Therefore, it should be accepted.

Non-local self-similarity in natural images has been well studied as an effective prior in image restoration. However, for single image super-resolution (SISR), most existing deep non-local methods (e.g., non-local neural networks) only exploit similar patches within the same scale of the low-resolution (LR) input image. Consequently, the restoration is limited to using the same-scale information while neglecting potential high-resolution (HR) cues from other scales. In this paper, we explore the cross-scale patch recurrence property of a natural image, i.e., similar patches tend to recur many times across different scales. This is achieved using a novel cross-scale internal graph neural network (IGNN). Specifically, we dynamically construct a cross-scale graph by searching k-nearest neighboring patches in the downsampled LR image for each query patch in the LR image.